Isotonic Classification Trees.

Nearest Neighbour Classification with Monotonicity Constraints.

Aligning Bayesian Network Classifiers with Medical Contexts.
While for many problems in medicine classification models are being developed, Bayesian network classifiers do not seem to have become as widely accepted within the medical community as logistic regression models. We compare first-order logistic regression and naive Bayesian classification in the domain of reproductive medicine and demonstrate that the two techniques can result in models of comparable performance. For Bayesian network classifiers to become more widely accepted within the medical community then, we feel that they should be better aligned with their context of application. We describe how to incorporate well-known concepts of clinical relevance in the process of constructing and evaluating Bayesian network classifiers to achieve such an alignment.
Nearest Neighbour Classification with Monotonicity Constraints.

Exceptional Model Mining.
Abstract. In most databases, it is possible to identify small partitions of the data where the observed distribution is notably different from that of the database as a whole. In classical subgroup discovery, one considers the distribution of a single nominal attribute, and exceptional subgroups show a surprising increase in the occurrence of one of its values. In this paper, we introduce Exceptional Model Mining (EMM), a framework that allows for more complicated target concepts. Rather than finding subgroups based on the distribution of a single target attribute, EMM finds subgroups where a model fitted to that subgroup is somehow exceptional. We discuss regression as well as classification models, and define quality measures that determine how exceptional a given model on a subgroup is. Our framework is general enough to be applied to many types of models, even from other paradigms such as association analysis and graphical modeling. 1
Nonparametric Monotone Classification with MOCA.

An Experimental Comparison of Different Inclusion Relations in Frequent Tree Mining.

Parameter Learning for Bayesian Networks with Strict Qualitative Influences.

Discriminative Scoring of Bayesian Network Classifiers a Comparative Study.
We consider the problem of scoring Bayesian Network Classifiers (BNCs) on the basis of the conditional loglikelihood (CLL). Currently, optimization is usually performed in BN parameter space, but for perfect graphs (such as Naive Bayes, TANs and FANs) a mapping to an equivalent Logistic Regression (LR) model is possible, and optimization can be performed in LR parameter space. We perform an empirical comparison of the efficiency of scoring in BN parameter space, and in LR parameter space using two different mappings. For each parameterization, we study two popular optimization methods: conjugate gradient, and BFGS. Efficiency of scoring is compared on simulated data and data sets from the UCI Machine Learning repository. 1
Learning Bayesian network parameters under order constraints.
We consider the problem of learning the parameters of a Bayesian network from data, while taking into account prior knowledge about the signs of influences between variables. Such prior knowledge can be readily obtained from domain experts. We show that this problem of parameter learning is a special case of isotonic regression and provide a simple algorithm for computing isotonic estimates. Our experimental results for a small Bayesian network in the medical domain show that taking prior knowledge about the signs of influences into account leads to an improved fit of the true distribution, especially when only a small sample of data is available. More importantly, however, the isotonic estimator provides parameter estimates that are consistent with the specified prior knowledge, thereby resulting in a network that is more likely to be accepted by experts in its domain of application. 1
Advances in Intelligent Data Analysis VI
The growing number of publicly available information sources makes it impossible for individuals to keep track of all the various opinions on one topic. The goal of our artificial believer system presented in this paper is to extract and analyze statements of opinion from newspaper articles. Beliefs are modeled using a fuzzy-theoretic approach applied after nlp-based information extraction. A fuzzy believer models a human agent, deciding what statements to believe or reject based on different, configurable strategies.
Advances in Intelligent Data Analysis VI
The growing number of publicly available information sources makes it impossible for individuals to keep track of all the various opinions on one topic. The goal of our artificial believer system presented in this paper is to extract and analyze statements of opinion from newspaper articles. Beliefs are modeled using a fuzzy-theoretic approach applied after nlp-based information extraction. A fuzzy believer models a human agent, deciding what statements to believe or reject based on different, configurable strategies.
Instability of Classifiers on Categorical Data.

Bringing order into bayesian%2Dnetwork construction.
Among the tasks involved in building a Bayesian network, obtaining the required probabilities is generally considered the most daunting. Available data collections are often too small to allow for estimating reliable probabilities. Most domain experts, on the other hand, consider assessing the numbers to be quite demanding. Qualitative probabilistic knowledge, however, is provided more easily by experts. We propose a method for obtaining probabilities, that uses qualitative expert knowledge to constrain the probabilities learned from a small data collection. A dedicated elicitation technique is designed to support the acquisition of the qualitative knowledge required for this purpose. We demonstrate the application of our method by quantifying part of a network in the field of classical swine fever.
Confidence intervals for probabilistic network classifiers.

Learning Bayesian Network Parameters with Prior Knowledge about Context%2DSpecific Qualitative Influences.

Confidence intervals for probabilistic network classifiers.

Monotonicity in Bayesian Networks.
In many realistic problem domains, the main variable of interest behaves monotonically in the observable variables, in the sense that higher values for the variable of interest become more likely with higher-ordered observations. This type of knowledge appears to naturally emerge from experts during knowledge elicitation, without explicit prompting from the knowledge engineer. The experts ’ concept of monotonicity, however, may not correspond to the mathematical concept of monotonicity in Bayesian networks. We present a method that provides both for verifying whether or not a network exhibits the properties of monotonicity suggested by the experts and for studying the violated properties with the experts. We illustrate the application of our method for a real Bayesian network in veterinary science. 1
Pruning for Monotone Classification Trees.
Abstract. For classification problems with ordinal attributes very often the class attribute should increase with each or some of the explanatory attributes. These are called classification problems with monotonicity constraints. Standard classification tree algorithms such as CART or C4.5 are not guaranteed to produce monotone trees, even if the data set is completely monotone. We look at pruning based methods to build monotone classification trees from monotone as well as nonmonotone data sets. We develop a number of fixing methods, that make a non-monotone tree monotone by additional pruning steps. These fixing methods can be combined with existing pruning techniques to obtain a sequence of monotone trees. The performance of the new algorithms is evaluated through experimental studies on artificial as well as real life data sets. We conclude that the monotone trees have a slightly better predictive performance and are considerably smaller than trees constructed by the standard algorithm. 1
Classification trees for problems with monotonicity constraints.
For classification problems with ordinal attributes very often the class attribute should increase  with each or some of the explaining attributes. These are called classification problems with  monotonicity constraints. Classical decision tree algorithms such as CART or C4.5 generally do  not produce monotone trees, even if the dataset is completely monotone. This paper surveys  the methods that have so far been proposed for generating derision trees that satisfy  monotonicity constraints. A distinction is made between methods that work only for monotone  datasets and methods that work for monotone and non-monotone datasets alike.
MAMBO Discovering Association Rules Based on Conditional Independencies.
We present the Mambo algorithm for the discovery of association rules. Mambo is driven by conditional independence relations between the variables instead of the minimum support restrictions of algorithms like Apriori. We argue that making use of conditional independencies is an intuitively appealing way to restrict the set of association rules considered. Since we only have a finite sample from the probability distribution of interest, we have to deal with uncertainty concerning the conditional independencies present. Bayesian methods are used to quantify this uncertainty, and the posterior probabilities of conditional independence relations are estimated with the Markov Chain Monte Carlo technique. We analyse an insurance data set with Mambo and illustrate the differences in results compared to Apriori.
A general model for automated business diagnosis.



Handling Missing Data in Trees Surrogate Splits or Statistical Imputation.
. In many applications of data mining a - sometimes considerable  - part of the data values is missing. This may occur because the  data values were simply never entered into the operational systems from  which the mining table was constructed, or because for example simple  domain checks indicate that entered values are incorrect. Despite the frequent  occurrence of missing data, most data mining algorithms handle  missing data in a rather ad-hoc way, or simply ignore the problem.  We investigate simulation-based data augmentation to handle missing  data, which is based on #lling-in #imputing# one or more plausible values  for the missing data. One advantage of this approach is that the imputation  phase is separated from the analysis phase, allowing for di#erent  data mining algorithms to be applied to the completed data sets.  We compare the use of imputation to surrogate splits, such as used in  CART, to handle missing data in tree-based mining algorithms. Experiments  show that im...
Mining in the Presence of Selectivity Bias and its Application to Reject Inference.

Bootstrapping and Validation of Metamodels in Simulation.
Bootstrapping is a resampling technique that requires less computer time than simulation does. Bootstrapping-like simulation- must be defined for each type of application. This paper defines bootstrapping for random simulations with replicated runs. The focus is on linear regression metamodels. The metamodel&#039;s parameters are estimated through Generalized Least Squares. Its fit is measured through Rao&#039;s lack-of-fit F-statistic. The distributions of this statistic is estimated through bootstrapping. The main conclusions are (i) not the regression residuals should be bootstrapped; instead the deviations that also occur in the standard deviation, should be bootstrapped (ii) bootstrapping Rao&#039;s lack-of-fit statistic is a good alternative to the F-test: it gives virtually identical results when the assumptions of the F-test are known to apply, and somewhat better results otherwise.
Data mining and related techniques.
Tracking Behavior Anomalies for identifying user impersonations is a field of active research. While various approaches have been used, it remains true that this process is still not deterministic or controllable enough for practical use. We present a new approach that aims to make the process of profiling behavior patterns of users deterministic by using a Rule-base to assign scores to various events. This also makes such a system highly controllable by permitting real-time modifications to the rule base. By combining asymmetric encryption with an XML based representation of behavior information, we have also made the profiling process a one-way task, where a compromised profile does not reveal sensitive information about the profiled user. We developed and tested a software system to evaluate the concept, and through this we have demonstrated how it is possible to ensure that reporting and monitoring components distributed over a network are not compromised, and our entire system functions on an XML based open, secure protocol. We also discuss the applicability of network packet capture as a source of behavior information. Our results indicate that we have been able to develop a deterministic, secure and extensible framework for distributed behavior anomaly detection. 1.
Learning from Biased Data Using Mixture Models.

Data Mining for Loan Evaluation at ABN AMRO A Case Study.

